1.1 Probability

#Measure #SigmaAlgebra

We have defined probability in STAT 201A note. However for a different course with (possibly) different notations, we will derive again.

1 What is Probability?

Frequentist answer: a relative frequency over many repeats of the same experiment.
Bayesian answer: a degree of belief that something is true/will happen.

Probability

Mathematical probability is a function $P$ mapping some subsets of a sample space $X$ to $[0, 1]$ , satisfying

$P (X) = 1$ ,
$P (\emptyset) = 0$ ,
(disjoint additivity) $P (⋃_{i = 1}^{\infty} A_{i}) = \sum_{i = 1}^{\infty} P (A_{i}), A_{i} \cap A_{j} = \emptyset, \forall i \neq j .$

2 Measure and Integrals

2.1 Measure

Given a set $X$ , a measure $μ$ is a certain kind of function mapping "nice enough" subsets $A \subset X$ to non-negative numbers $μ (A) \in [0, + \infty)$ .

Examples of Measure

Counting measure: $X$ is countable (like $Z$ ), use $# (A)$ to define the number of points in $A$ .
Lebesgue measure: $X = R^{n}$ , $λ (A) = Volume (A) = \int \dots \int_{R^{n}} 1_{A} d x_{1} d x_{n}$ .
Gaussian measure: $X = R$ , $P_{Z} (A) = P (Z \in A) = \int_{A} ϕ (x) d x .$

Generally, the domain of a measure is just a collection of "nice" subsets $F \subset 2^{X}$ . $F$ should satisfy certain closure properties.
First we need definition of $σ -$ algebra. It's not important. See definition here.

Measure

Given measurable space $(X, F)$ , a measure $μ$ is a map $F \to [0, + \infty]$ with disjoint additivity and $μ (\emptyset) = 0$ . $μ$ is probability measure if $μ (X) = 1$ .

2.2 Integral

Integral

An integral w.r.t $μ$ puts weight $μ (A)$ on $A$ . Define $\int_{A} 1 {x \in A} d μ (x) = μ (A) .$ We can extend to other functions by linearity $\int c 1_{A} (x) d μ (x) = c μ (A) \Rightarrow \int \sum_{i = 1}^{n} c_{i} 1_{A_{i}} (x) d μ (x) = c \sum_{i = 1}^{n} μ (A_{i}),$
and limits $\int f d μ = lim_{n \to \infty} \int f_{n} (x) d μ .$
Pasted image 20241201225736.png|600

Example

For counting measure, $\int_{X} f d # = \sum_{x \in X} f (x)$ .
For Lebesgue measure, $\int_{X} f d λ = \int \dots \int_{X} f (x) d x_{1} \dots d x_{n}$ .
For Gaussian measure, $\begin{aligned} \int 1_{A} (x) d P_{Z} (x) = P_{Z} (A) = \int_{- \infty}^{+ \infty} 1_{A} (x) ϕ (x) d x \\ \Rightarrow & \int f (x) d P_{Z} (x) = \int_{- \infty}^{+ \infty} f (x) ϕ (x) d x = E [f (Z)] . \end{aligned}$

3 Densities

A measure $P$ is absolutely continuous w.r.t $μ$ , if $P (A) = 0$ whenever $μ (A) = 0$ . Denote as $P ≪ μ$ or $μ$ dominates $P$ .
If $P ≪ μ$ , then define density function $p : X \to [0, + \infty)$ s.t. $P (A) = \int 1_{A} (x) p (x) d μ (x), \forall A \in F,$ and by extension $\int f (x) d P (x) = \int f (x) p (x) d μ (x)$ . Density function $p$ is also called Radon-Nikodym derivative of $P$ w.r.t $μ$ , and is sometimes written as $\frac{d P}{d μ} (x)$ .

If we don't specify $μ$ , it is the Lebesgue measure. I.e., $P$ is absolutely continuous in default means $P ≪ λ$ .

If $P$ is probability, and $μ$ is Lebesgue measure, then $p$ is called probability density function (p.d.f);
Elif $μ$ is counting measure, $p$ is called probability mass function (p.m.f).

4 Probability Spaces, Random Variables

Denote the outcome space $Ω$ . To evaluate whatever $P (X, \dots)$ , it is convenient to start with abstract outcome $ω \in Ω$ .

$ω$ represents everything that "happens".
Quantities of interest are functions of $ω$ . (i.e., $X (ω)$ )

Probability Space

We have $(Ω, F, P)$ a probability space.

$ω \in Ω$ is called outcome.
$A \subset Ω$ is called event.
$P (A)$ is called probability of event $A$ .
Function $X : Ω \to X$ is called random variable $X (ω)$ .
We say $X$ has distribution $Q$ (defined as $X \sim Q$ ) if $P (X \in B) = Q (B)$ .

$Q (B)$ is the push-forward of $P$ through function $X (ω)$ : $Q (B) \equiv P \circ X^{- 1} (B)$ .

Applied more generally, if $μ$ is a measure on $X$ , $f : X \to Y$ leads to new measure $\underset{\in Y}{\underset{⏟}{ν (B)}} = μ \circ \underset{\in X}{\underset{⏟}{f^{- 1} (B)}}$ . ( $f^{- 1}$ denotes the preimage)

If $P (A) = 1$ , we say $A$ happens almost surely.